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Abstract 

Students in an undergraduate human development course (N = 215) participated in a brief 
assessment of their reading (comprehension level, reading speed, comprehension rate) and 
multiple-choice test-taking skills on the second day of class. Students first read a one-page, 
400-word passage unrelated to the course and then answered 10 multiple-choice questions 
over the passage without referring back to the passage. To control for test-taking skills, 
students also answered 10 multiple-choice questions from an equivalent passage they did not 
read. Videotapes of student participation permitted individual assessment of time required to 
complete each phase. Subsequently, during the semester students took five 50-item multiple- 
choice exams over the major units in the course. Results showed that the brief reading 
comprehension measures predicted multiple-choice exam performance and that 
comprehension level accounted for most of the variance in exam performance. Discussion 
focuses on enhancing brief reading assessment procedures by including direct measures of 
comprehension. 
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Extending Research on the Validity of Brief Reading Comprehension Rate and Level Measures 

to College Course Success 

Several measures predict success in college courses, including measures of critical 
thinking, generic vocabulary, background knowledge, and reading comprehension (Behrman 
& Street, 2005; Jackson, 2005; Williams & Eggert, 2002; Williams, Oliver, Allin, Winn, & 
Booher, 2003a; Williams, Oliver, Allin, Winn, & Booher, 2003b; Williams, Oliver, & Stockdale, 
2004; Williams & Worth, 2002). Not all course activities require high-level critical thinking, an 
advanced vocabulary, or background knowledge, but virtually all course activities require 
student reading of course material (Behrman & Street, 2005). Thus, the current study 
focused on reading comprehension measures as predictors of success in a large 
undergraduate course. 

Although reading comprehension is predictive of student success (Jackson, 2005), 
reading speed or fluency may also be related to comprehension levels, effort required to 
read, and reinforcement for reading (Skinner, Pappas, & Davis, 2005). Researchers have 
measured reading fluency by timing students' oral reading and scoring word accuracy. These 
data are then used to calculate words correct per minute (WCM), which has been shown to 
correlate with reading comprehension and other reading skills as measured via standardized, 
norm-referenced tests with strong psychometric properties (Deno & Merkin, 1977; Deno, 
Merkin, & Chiang, 1982; Fuchs, & Fuchs, 1992; Fuchs, Fuchs, Hosp, & Jenkins, 2001; Fuchs, 
Fuchs, & Maxwell, 1988; Flintze & Shapiro, 1997; J enkins & J ewell, 1993). These studies 
provide support for several theories that suggest causal mechanisms for explaining the 
relationship between reading speed and comprehension. 

Students who read rapidly and are not required to apply their attention or other 
cognitive resources (e.g., working memory) towards decoding words have more cognitive 
resources available to apply towards comprehension. Additionally, as time passes information 
may become inaccessible (e.g., fading from working memory). Thus, as they read, rapid 
readers may have access to more information from material read earlier than slow readers, 
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which may enhance their ability to synthesize information as they progress through passages 
(LaBerge & Samuels, 1974; Perfetti, 1992; Rasinski, 2004; Stanovich, 1986). 

Reading, like many other skills, improves as people choose to spend more time 
engaged in the activity (Daly, Chafouleas, & Skinner, 2005; Stanovich, 1986). Even when 
slower readers are able to comprehend the material at the same level as rapid readers, rapid 
readers are more likely to choose to read the assigned work because it requires less time and 
effort and results in a higher rate of reinforcement (Billington, Skinner, Hutchins, & Malone, 
2004; Skinner, 1998; Skinner, 2002; Skinner, Neddenriep, Bradley-Klug, & Ziemann, 2002; 
Skinner, Wallace, & Neddenriep, 2002). Additionally, fluent readers have more time to 
engage in other behaviors that may enhance learning (e.g. , studying notes) and test 
performance (e.g., carefully considering all responses on a multiple choice exam) than those 
who require more time to read (Skinner et al. , 2005). 

Although the psychometric research base supporting WCM is strong, the correlations 
with broad reading skill development begin to decline as skills develop beyond the 4th or 5th 
grade level (Hintze & Shapiro, 1997; J enkins & J ewell 1993; Neddenriep, Skinner, Hale, 

Oliver, & Winn, in press). Jackson's (2005) comparison of reading comprehension, oral 
reading fluency, and decoding showed that reading comprehension was the only significant 
predictor of college students' grade point average. Thus, reading comprehension, rather than 
the ability to read aloud both accurately and rapidly, may be the most essential reading skill 
for older students (Chall, 1983; Fuchs & Fuchs, 1992; Potter & Wamre, 1990; Shapiro, 2004; 
Skinner, Neddenriep et al., 2002). However, because college students are often given a 
limited amount of time to complete numerous assigned readings and exams, reading speed 
also may predict success in college courses. 

Recently, researchers have developed and evaluated a measure of reading 
comprehension rate (Hale, Skinner, Williams, Neddenriep, & Dizer, in press; Neddenriep et 
al., in press; Skinner, 1998). Reading comprehension rate (RCR) is similar to WCM in that it 
takes into account reading speed, using time required to read the material in the 
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denominator. However, the numerator (number of words read aloud and accurately) is 
replaced with a measure of reading comprehension level (percent correct on comprehension 
questions). This RCR measure can be converted to a common metric, percent passage 
comprehended for each minute spent reading (Skinner, 1998). 

Neddenriep et al. (in press) correlated reading comprehension level and RCR with 
broad reading cluster scores on the Woodcock-] ohnson III Tests of Achievement (Woodcock, 
McGrew, & Mather, 2001). Results showed that silent reading comprehension level (percent 
comprehension questions correct) moderately correlated with 10th grade readers' broad 
reading scores (r= .40). When this measure was converted to a RCR measure, the 
correlation increased ( r= .53). These results showed that altering the traditional 
comprehension level measure to a comprehension rate measure enhanced the ability to 
predict broad reading skills. Others have shown that RCR is a sensitive measure of subtle 
differences in reading comprehension occasioned by different intervention procedures 
(Freeland, J ackson, & Skinner, 1999; Freeland, Skinner, J ackson, McDaniel, & Smith, 2000; 
Hale et al., in press; Hale, Skinner, Winn, Oliver, Allin, & Molloy, 2005; McDaniel, Watson, 
Freeland, Smith, J ackson, & Skinner, 2001). 

Researchers concerned with measuring reading fluency by assessing rate of aloud 
reading accuracy have suggested that measuring RCR (e.g., percent of passage understood 
for each minute spend reading) may provide a more direct measure of functional reading 
skills, especially in advanced readers (Skinner, 1998; Skinner, Neddenriep, et al., 2002). The 
current study was designed to extend the research on the validity of brief assessments of 
reading comprehension levels and rates by determining if these measures could predict a 
functional outcome for skilled readers (i.e., success in a large college course). We controlled 
for test-tasking skills (Stough, 1993) by requiring students to answer questions covering 
passages that they had not read. Analysis included performance on factual and inferential 
comprehension questions. Finally, we also examined the relationship between reading speed 
and comprehension. 
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Method 

Participants 

All students came from five sections of an undergraduate human development course 
required for entry into the teacher-education program at a major state university in the 
Southeastern United States. Three large sections had approximately 55 students per section 
and two small sections had approximately 25 students per section. Because a few students 
dropped the course, 210 students out of the original 215 ultimately completed all the 
measures related to the study. Approximately 25% of the students were males and 75% 
females. With respect to students' academic level, 3% were freshmen, 40% sophomores, 
30% juniors, 17% seniors, and 10% graduate students. 

Research Materials and Procedures 

When students entered the classroom on the second day of the course, they found a 
packet of materials on their desk. After students sat down, they first displayed their name 
card on the desk immediately in front of them. Then they filled in selected demographic 
information (e.g., academic classification, sex, and course section) on the front sheet of the 
materials packet, but did not open the rest of the packet until instructed to do so. When 
instructed to turn to the second page of the packet, they saw the following information: 

On the following page, you will find a reading passage. When instructed, please turn to the 
passage and read it silently. Read at your normal rate— neither faster nor slower than usual. 
However, read carefully because you will be asked questions about the passage. Raise your hand 
and keep it raised for three seconds when you have finished reading the passage. 

A 400-word passage taken from Spargo (1989) appeared on the third page of the 
packet. The selected passage, entitled "More Rare than Rubies," mainly dealt with the 
contemporary importance of rare heavy metals and the methods of prospecting for these 
metals (p. 43). The course did not deal with heavy metals; plus, students were unlikely to 
have much background information regarding heavy metals. After finishing the passage and 
raising their hand for three seconds, students then proceeded to the next page. This page 
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contained the following instructions: 

On the following page, you will find comprehension questions based on the passage you just 
read. Try your best to answer these questions accurately. Do not refer back to the passage when 
answering the questions. When you have finished answering the questions, raise the packet of 
materials in your hand and keep it raised for three seconds. 

The subsequent page contained ten multiple-choice questions, with three options per 
questions. Five of the questions were labeled "Recalling Facts” and five "Understanding the 
Passage." The latter questions required inferences from the factual information in the 
passage. The bottom of this page repeated the instruction to raise the packet and keep it up 
for three seconds after completing the questions and before proceeding to the next page. 

After turning to the next page, students read instructions related to answering 
questions over a passage they had not read: 

On the following page, you will find comprehension questions based on a passage that you have 
not read. Try your best to answer these questions accurately. When you have finished answering 
the questions, raise your hand and keep it raised for three seconds. 

The following page contained ten multiple-choice questions, with three options per question, 
based on passage entitled "Sewerage Disposal” from Spargo (1989, p.91). Five questions 
were factual and five were inferential. Again, the course did not deal with the information in 
this passage and students were unlikely to have much background information regarding the 
content in the passage. Thus, their performance would likely be based on chance or cues 
embedded in the questions themselves. The bottom of this page included instructions 
reminding the students to raise their hand and keep it raised for three seconds after 
completing the questions and before turning the page. The final page instructed the students 
to place their packet face down on the desk in front of them and wait for instructions 
regarding the next research activity. 

In order to precisely calibrate how much time each student took to complete each 
phase of the data collection, we used multiple video cameras to record the hand/package 
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signals of every student in the classroom. The combination of students' positioning name 
cards on the desk in front of them and raising their hands and/or packets when they finished 
each phase of the data collection permitted precise determination of when each student 
began and ended each phase. In viewing the videotapes, research associates used stop 
watches to make time determination by the second. 

In addition to the pre-course data collection, we administered five 50-item multiple 
choice exams (four options per item) over the five units in the course: physical development, 
cognitive development, psychological development, social development, and character 
development. Previous research (Wallace & Williams, 2003) on the exam items has indicated 
that about 26% of the items strictly assessed recall of factual information, 58% required use 
of factual information to make inferential judgments, and 16% did not exclusively fit either 
the factual or inferential category. All three item types differentiated between high and low 
performers on the course exam. 

Results 

Presentation of the results first highlights the relationship between reading and test- 
taking variables to performance on the course multiple-choice exams. Next, we present the 
results of our analysis of passage reading speed and comprehension. Finally, high and low 
performers on the exams were compared across reading and test-taking variables. 

Exam Performance and Reading Measures 

Table 1 shows that virtually all of the reading-comprehension variables correlated 
significantly with exam performance. RCR was calculated using two terms, total questions 

Insert Table 1 About Here 

correct in the numerator and reading time in the denominator. The significant negative 
correlation ( i.e. , r = -.19,; i 2 = .04) between passage reading time and exam performance 
suggests that reading speed is related to performance in college course work. However, the 
numerator, comprehension questions correct, accounted for more variance (r = .37; r 2 = 
.14) in exam scores than the denominator, reading time (r 2 = .04). Furthermore, much of 
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the variance in exam performance accounted for by reading speed is shared with the 
variance accounted for by the total reading comprehension level measure (total questions 
correct). Thus, when these two measures are combined, RCR correlated only slightly higher 
(r=. 39; i 2 = .15) with exam performance than reading comprehension level (r = .37, i 2 = 
.14). 

When number correct on the five factual items was compared with number correct 
on the inferential items as predictors of course exam performance, the inferential score 
accounted for 10% of the variance in exam scores (r =.31, i 2 = .10) and factual scores 
accounted for 4% of the variance [r=. 21, i 2 = .04). The combination of the factual and 
inferential scores accounted for 14% of the variance in exam scores. Interestingly, this 
analysis showed that factual and inferential comprehension did not account for the same 
variance in exam score performance. 

Comparisons of time taken to answer the questions across passages showed that 
students spent significantly, 7(214) = 13.18, p < .001, more time answering the non-passage 
items (x = 167.84 seconds) than the passage items (x = 133.60 seconds) relative. These 
comparisons suggest that students attempted to apply their test-taking skills to answer the 
non-passage items. However, Table 1 also shows that none of the pre-course exam scores 
based on an unread passage correlated significantly with course exam performance. Even for 
the reading passage, time taken to answer the questions and rate of correctly answering the 
questions did not correlate significantly with performance on the course exams. Thus, test- 
taking skills were ruled out as a predictor of exam performance in the course and as a 
plausible rival hypothesis accounting for the relationship between reading variables and exam 
scores. 

Passage Reading Time and Passage Comprehension Measures 

Table 2 shows the correlations between the three passage-reading-comprehension 
measures (i.e., factual items correct, inferential items correct, and total correct) and passage 

insert Table 2 About Here 
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reading time. Results show a significant correlation ( p < .05) between total questions correct 
and reading time (r = -.161). This small negative correlation between comprehension level 
and reading time explains why converting reading comprehension level to a rate measure 
(i.e., incorporating time to read) did not account for much more variance than reading 
comprehension alone. Table 2 also shows a significant correlation (p < .05) between 
inferential reading comprehension and reading time (r = -.160), but an insignificant 
relationship between factual questions correct and reading time (r= -.073). These results 
show that students who spent less time reading answered more inferential passage questions 
correctly than students who spent more time reading. 

Mean Comparisons 

Students scoring in the top and bottom 20% on the unit exam totals were compared 
on all the reading variables. Table 3 shows significant differences on the pre-course 
performance measures linked to the read passage, but not on those linked to the unread 
passage. Additionally, 

Insert Table 3 About Here 

the high and low exam performers did not differ in the time taken to answer the questions 
over the reading passage or in the rate of correctly answering these questions. Although the 
effect sizes for total reading comprehension and RCR were very strong, the effect size for 
RCR (2.00) was almost twice as great as for reading comprehension level (1.17). Thus, at the 
performance extremes on the course exams, high-performing students clearly differed from 
the low-performing on the reading comprehension measures, most especially RCR, but not 
on the test-taking variables. The ability to comprehend reading material, especially at a more 
rapid rate, apparently facilitated exam performance but test-taking pace and skills per se did 
not significantly predict exam performance. 

Discussion 
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Neddenriep et al. (in press) found that brief RCR measures correlated with standardized 
measures of broad reading skill development in elementary and 10th grade students. We 
extended this research to college students and showed that RCR could predict performance 
on college exams, a more functional criterion measure than performance on standardized 
tests. Overall, relationships between the reading variables and performance on the course 
exams ranged from small to medium, but comparisons of high and low exam performers 
showed substantial differences on the reading variables but no significant differences on the 
test-taking pace and skill variables. These results suggest that the reading measures, not 
test-tasking skills, account for these relationships. These findings may have theoretical and 
applied implications. 

RCR measures include both a measure of reading speed (denominator) and 
comprehension (numerator). Some theorists, both behavioral (e.g., Skinner, 1998; Skinner, 
Wallace et al., 2002) and cognitive (e.g., LaBerge & Samuels, 1974; Perfetti, 1992) have 
proposed causal models describing the relationship between reading speed or fluency and 
the development of reading skills that are functional (i.e., comprehension level and rate). 
These theories and the research base supporting these theories have caused educators, 
researchers, and policy makers to recommend that educators implement procedures to 
enhance reading speed (e.g., Adams, 1990; Daly, Chafouleas, & Skinner, 2005; National 
Assessment of Educational Progress, 2005; Rasinski, 2004). Several of our findings support 
these theories and recommendations. The time students spent reading the passage was 
negatively correlated (r- -.16) with passage comprehension (total questions correct). Thus, 
students who spent less time reading answered more questions correctly. Second, the 
significant correlation ( r= -.19) between reading time and exam performance showed that 
more rapid reading is associated with higher performance on course exams. Compared to 
factual questions, inferential questions may require students to synthesize more information 
across the text. The significant negative correlation between reading time and inferential 
questions answered correctly supports the theory that slower readers may have had difficulty 
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synthesizing information across the passage because information dropped from working 
memory as time passed. 

The current results support the relationship between reading speed and success in 
college courses. However, almost all of the variance in students' exam scores accounted for 
by the RCR measure can be attributed to the measure of reading comprehension 
(numerator), as opposed to reading time (denominator), embedded within the RCR measure. 
These results support previous researchers who have stressed the need to alter brief 
assessments of reading skills in advanced readers by incorporating direct assessment of 
reading comprehension level and rate (Jackson, 2005; Neddenriep et al. , in press; Skinner et 
al. , 2002). The current results extend the research base by suggesting that the direct 
assessment of comprehension, as opposed to speed, is most critical for predicting success in 
college courses. 

Our results with college students are consistent with the Neddenriep et al. (in press) 
finding that RCR better predicted overall reading skill than reading comprehension level in 
lOth-grade students. However, our results differed in that RCR correlated only slightly higher 
(r = .39) with exam scores than reading comprehension level (r = .37), whereas Neddenriep 
et al. found greater differences between the predictive ability of silent reading 
comprehension level (r = .40) and silent RCR (r = .53). Several differences between the 
current study and the Neddenriep et al. study may account for the stronger support for RCR 
in the Neddenriep study. The current study included a sample of college students, as 
opposed to 10 th -grade students. A group of college students is likely to have less variability in 
their reading skills than a group of 10 th -grade students, thus reducing the predictive potential 
of reading scores for college students. Also, the criterion variable differed across the two 
studies, with Neddenriep et al. using a standardized measure of broad reading ability and the 
current researchers using exam scores in a college class. Additional studies are needed to 
determine if RCR is a more significant indicator of overall reading skills, but reading 
comprehension alone may be sufficient to adequately predict classroom performance. 
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The current results showed that correctly answering inferential questions over what one 
has just read was more predictive of exam performance than correctly answering factual 
questions. A majority of the items on the course exams required some degree of inference 
about course information (Wallace & Williams, 2003), suggesting that test-text overlap may 
have enhanced the correlations. Perhaps some of the predictive potential was that both the 
pre-course assessment and the in-course examinations used a multiple-choice format. 
However, one must keep in mind that performance on the pre-course multiple-choice 
questions linked to an unread passage was not predictive of course exam performance. 
Regardless, additional research is needed to determine if reading comprehension question 
type (fact versus inference, multiple choice versus essay) influences the predictive validity of 
reading comprehension measures across various criteria measures (e.g., multiple-choice 
exams, essay exams, standardized reading achievement tests, test of critical thinking). 

Individually administering lengthy standardized measures of reading comprehension 
at the beginning of courses may strengthen the relationships between reading skill 
assessment and exam performance. However, given that time typically is at a premium in 
large undergraduate courses, devoting a significant amount of time to pre-course assessment 
would be a questionable use of class time. Group administration of the reading 
comprehension measures used in the current study required much less time (the student 
requiring the most time needed 13.92 minutes) to get a sense of how well students were 
likely to do on major course exams across the semesters. Actually, it appears quite 
remarkable that an activity requiring less than 15 minutes for the slowest-working student 
could generate data predictive of performance on course exams spanning an entire semester. 
Regardless, it is very easy to group administer and score comprehension level, but more 
difficult and time consuming to collect reading speed data ( i. e. , taping all group members, 
viewing the tapes, and recording each students' time spent reading). Our results suggest that 
collecting data on reading speed may not be necessary, as including reading time results in 
only a small increase in predictive power. Future research may find that they can efficiently 
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strengthen the predictive power associated with group administration of silent reading 
comprehension level measures by using longer passages, passages with more questions, 
and/or or multiple passages with the median scores used as the predictor (Skinner et al., 
2002 ). 

The findings of the current study challenge a claim often advanced by our students who 
do poorly on the multiple-choice exams in the target course. Anecdotally, a rather common 
claim among these students is that they understood the material really well but the tests did 
not allow them to demonstrate their understanding. An extension of this claim is that the 
course exams primarily measure skill in taking multiple-choice tests rather than 
understanding of course content. However, the purest measure of test-taking skills in the 
current study (answering multiple-choice questions over a passage not read) was not related 
to performance on the multiple-choice exams in the course, even at the extremes of exam 
performance. In contrast, the ability to answer multiple-choice questions over a passage just 
read was predictive of exam performance. 

Summary 

Skinner (1998) indicated that reinforcement for reading is typically contingent upon 
comprehension (i.e., the function of reading is comprehension). The most commonly used 
brief measure of oral reading fluency is WCM (Daly et al., 2005). Although WCM correlates 
with reading comprehension, using correlates to indirectly measure anything requires a level 
of inference that may lead to questionable outcomes. For example, educators may attempt to 
apply procedures that enhance aloud reading speed but do not necessarily enhance reading 
comprehension. Thus, a more direct measure of functional reading skills is a measure of 
silent reading comprehension level and/or rate (Skinner et al., 2002). In addition to 
conducting additional studies designed to investigate and enhance the psychometric 
properties of silent reading comprehension measures (e.g., validity, reliability, sensitivity), 
researchers should continue to use these measures to assess the effects of interventions 
designed to enhance silent reading comprehension, the most direct measure of functional 
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